Missing the Point: Isolation Alone Will Not Build AI Safety
Treating AI agents like contract nurses and minimizing their access to critical patient information certainly reduces potential risk. But, it also minimizes key opportunities.
Treating AI agents like contract nurses and minimizing their access to critical patient information certainly reduces potential risk. But, it also minimizes key opportunities.
As generative AI moves toward performing increasingly consequential clinical functions, the American Medical Association (AMA) is urging the U.S. Food and Drug Administration to address a foundational question: Who is responsible for ensuring that these tools remain safe once they enter clinical practice?
Rick Abramson, a radiologist by training and the director of FDA's Digital Health Center of Excellence said, "We ended up putting out this idea that maybe these devices are starting to behave a little bit more like me as a physician than a heart valve or a defibrillator."
Many hospitals are now trying to use AI to automate processes and free up staff time, running simultaneous pilots to perform as many tasks as possible. This quickly leads to pilot sprawl — a state where organizations can no longer even say which agents are still in active use.
The Oracle Health Clinical AI Agent is unique in the way it’s embedded directly in the Oracle Health Foundation EHR, making it easier for nurses who don’t have to shift between tools used for separate tasks.
Imprivata CEO Fran Rosch said AI agents should be treated similarly to the way hospitals treat a new contract nurse — vetted, monitored and then cut off as soon as the job is done.
Epic faced an FTC antitrust probe this week, but that didn't stop the EHR giant from unveiling a wave of new AI tools at its annual conference.
Most healthcare organizations are experimenting with AI. Few are preparing to manage AI agents as participants in everyday healthcare workflows.
In this episode of the MedCity Pivot Podcast, CVS Health Aetna's chief digital and technology officer talks about how the insurance business is using AI, especially agentic AI, in healthcare. Point to note: AI is not used in automatic denials of coverage.
Grove AI’s technology uses voice agents to recruit eligible patients for clinical trials. Formed in 2024, the startup’s rapid growth led to its acquisition by Hippocratic AI earlier this year.
Assured Health raised $19 million for its AI agents that verify provider credentials and manage insurance enrollment for health systems and group practices. The startup said it can cut a process that usually takes months down to days.
We are moving into an era where we can finally converse with our data. We can ask, "Why are denials spiking in Region X?" or "Show me the safety profile of this cohort," and receive an answer, not a spreadsheet.
Penn Medicine is integrating K Health’s AI-powered patient intake agents into its virtual primary care service. The health system says the goal is to free up time for patient care rather than information-gathering.
Patients are trying to navigate insurance confusion, coordinate care, manage prescriptions, and understand the following steps that often feel unclear. Static chatbots were never built for this level of responsibility. Agentic AI is. Or at least that is the idea.
These three patterns keep showing up, and each is much cheaper to prevent than to fix later.